Image processing system, image processing device and image processing program

ABSTRACT

An image processing system, an image processing device and an image processing program facilitating setting of illumination condition are provided. A control device controls a light emission portion in a manner that each of plural types of partial regions set on a light emission surface emits light, and controls a camera to image an object in synchronization with light emission of each of the plural types of partial regions. The control device outputs reflection profile information, which is generated based on input images generated in each partial region emitting light, and the reflection profile information shows relationships between positions within the light emission surface and degrees of light reflected at attention sites of the object and incident to the camera with respect to light irradiated from the positions to the object.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the priority of Japan patent application serialno. 2018-047628, filed on Mar. 15, 2018. The entirety of theabove-mentioned patent application is hereby incorporated by referenceherein and made a part of this specification.

BACKGROUND Technical Field

The disclosure relates to an image processing system, an imageprocessing device and an image processing program.

Related Art

In an FA (Factory Automation) field and the like, an image processingtechnique is used in which an object is imaged under illumination bylight from an illumination device and information about a work-piece isobtained from image data that is generated.

As the illumination device used in an image processing technology field,various illumination devices are developed. For example, in Japaneselaid-open No. 2015-232487 (patent document 1), an illumination deviceincluding a plurality of lamps with different illumination directions isdisclosed.

Related Art Document(s) Patent Document(S)

(Patent Document 1) Japanese Laid-Open No. 2015-232487

When the illumination device including a plurality of lamps withdifferent illumination directions is used, there is a risk that whenillumination conditions are determined, there are many patterns ofselectable lamps and setting of the illumination conditions becomecomplicated.

SUMMARY

According to one example of the disclosure, an image processing systemwhich has an imaging portion imaging an object and a light emissionportion having a light emission surface directed toward the object isprovided. The image processing system includes: a light emission controlportion, which controls the light emission portion in a manner that eachof plural types of partial regions set in advance in the light emissionsurface emits light; an imaging control portion, which controls theimaging portion to image in synchronization with light emission of eachof the plural types of partial regions; and an output portion, whichoutputs reflection profile information, wherein the reflection profileinformation is obtained based on a plurality of images which arecaptured by the imaging portion in synchronization with the lightemission of each of the plural types of partial regions, and thereflection profile information shows relationships between positionswithin the light emission surface and degrees of light reflected atattention sites of the object and incident to the imaging portion withrespect to the light irradiated to the object from the positions.

According to another example of the disclosure, an image processingdevice which controls an imaging portion imaging an object and a lightemission portion having a light emission surface directed toward theobject to perform an image processing is provided. The image processingdevice includes: a light emission control portion, which controls thelight emission portion in a manner that each of plural types of partialregions set in advance in the light emission surface emits light; animaging control portion, which controls the imaging portion to image insynchronization with light emission of each of the plural types ofpartial regions; and an output portion, which outputs reflection profileinformation, wherein the reflection profile information is obtainedbased on a plurality of images which are captured by the imaging portionin synchronization with the light emission of each of the plural typesof partial regions, and the reflection profile information showsrelationships between positions within the light emission surface anddegrees of light reflected at attention sites of the object and incidentto the imaging portion with respect to the light irradiated to theobject from the positions.

According to another example of the disclosure, an image processingprogram which is executed in an image processing device is provided, andthe image processing device controls an imaging portion imaging anobject and a light emission portion having a light emission surfacedirected toward the object to perform an image processing. The imageprocessing program includes: controlling the light emission portion in amanner that each of plural types of partial regions set in advance inthe light emission surface emits light; controlling the imaging portionto image in synchronization with light emission of each of the pluraltypes of partial regions; and outputting reflection profile information,wherein the reflection profile information is obtained based on aplurality of images which are captured by the imaging portion insynchronization with the light emission of each of the plural types ofpartial regions, and the reflection profile information showsrelationships between positions within the light emission surface anddegrees of light reflected at attention sites of the object and incidentto the imaging portion with respect to the light irradiated to theobject from the positions.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram schematically showing an application scene of animage processing system of an embodiment.

FIG. 2 is a diagram schematically showing reflection profileinformation.

FIG. 3 is a schematic diagram showing one example of a production linein which the image processing system of the embodiment is applied.

FIG. 4 is a schematic diagram showing a hardware configuration of acontrol device.

FIG. 5 is a schematic diagram in which one portion of an illuminationdevice of the embodiment is enlarged.

FIG. 6 is a diagram showing one example of a process for obtainingreflection profile information in a first specific example.

FIG. 7 is a diagram for describing an obtainment method of data.

FIG. 8 is a diagram for describing an extraction method of the data.

FIG. 9 is a diagram for describing a generation method of the reflectionprofile information.

FIG. 10 is a diagram showing one example of an output method of thereflection profile information.

FIG. 11 is a diagram showing one example of a user interface screen atthe time of setting.

FIG. 12 is a diagram for describing one example of a generation methodof an inspection image.

FIG. 13 is a diagram schematically showing a function configuration ofthe control device.

FIG. 14 is a flowchart of processing of outputting the reflectionprofile information of a designated attention position.

FIG. 15 is a diagram for describing relative positions.

FIG. 16 is a diagram showing a generation method of reflection profileinformation in a second specific example.

FIG. 17 is a diagram showing an example of outputting the reflectionprofile information in the second specific example as mapping images.

FIG. 18 is a diagram for describing a determination method of anirradiation pattern of every attention position using a referenceirradiation pattern.

FIG. 19 is a diagram for describing a generation method of theinspection image using the reference irradiation pattern.

FIG. 20 is a diagram showing a user interface screen at the time ofsetting the reference irradiation pattern.

FIG. 21(a) to FIG. 21(d) are diagrams showing one example of a flow ofsetting the reference irradiation pattern.

FIG. 22 is a diagram schematically showing a function configuration ofthe control device in the second specific example.

FIG. 23 is a flowchart of processing of outputting the reflectionprofile information in the second specific example of the designatedattention position.

FIG. 24 is a diagram showing a variation example of a positionrelationship among the illumination device, a camera and an object.

FIG. 25 is a diagram showing a variation example of the illuminationdevice.

DESCRIPTION OF THE EMBODIMENTS

The disclosure provides an image processing system, an image processingdevice and an image processing program, which facilitates the setting ofthe illumination conditions as described above.

According to one example of the disclosure, an image processing systemwhich has an imaging portion imaging an object and a light emissionportion having a light emission surface directed toward the object isprovided. The image processing system includes: a light emission controlportion, which controls the light emission portion in a manner that eachof plural types of partial regions set in advance in the light emissionsurface emits light; an imaging control portion, which controls theimaging portion to image in synchronization with light emission of eachof the plural types of partial regions; and an output portion, whichoutputs reflection profile information, wherein the reflection profileinformation is obtained based on a plurality of images which arecaptured by the imaging portion in synchronization with the lightemission of each of the plural types of partial regions, and thereflection profile information shows relationships between positionswithin the light emission surface and degrees of light reflected atattention sites of the object and incident to the imaging portion withrespect to the light irradiated to the object from the positions.

According to the disclosure, the reflection profile information which isa reference in determining illumination conditions is output, so thatthe setting of the illumination conditions becomes easy.

In the above-described disclosure, the reflection profile informationmay be information which is obtained from each of the plurality ofimages and which is based on luminance information corresponding toattention points of an imaging visual field in the images and relativepositions of the partial regions in which the light is emitted when theimages are captured with respect to the attention points.

According to the disclosure, the reflection profile information obtainedfor every attention point can be compared with one another in the samedimension regardless of the position relationships between the lightemission surface and these attention points.

In the above-described disclosure, the output portion may output thereflection profile information by a representation form in which theinformation corresponding to the luminance information is output to acoordinate system with two or more axes corresponding to the relativepositions.

According to the disclosure, regardless of position relationship betweenthe light emission surface 40 and the attention points, they areexpressed in the same coordinate system, so that comparison can be madeeasily when comparing the reflection profile information for everyattention point.

In the above-described disclosure, the output portion may output thereflection profile information by the representation form in which theinformation corresponding to the luminance information is input to thecoordinate system with two or more axes corresponding to the relativepositions, wherein the reflection profile information corresponds to theattention points that are determined based on the position informationon the images which are designated by a user with respect to the imagesof the imaging visual field imaged by the imaging portion.

According to the disclosure, the user can easily designate theseattention points.

In the above-described disclosure, the output portion may simultaneouslyor sequentially output the reflection profile information obtained for aplurality of attention points within the imaging visual field to thecoordinate system.

According to the disclosure, the reflection profile information of everyattention point is simultaneously or sequentially output to the samecoordinate system, so that it is easy to compare the reflection profileinformation of every attention point.

In the above-described disclosure, the image processing system mayfurther include a determination portion which determines light emissionconditions of the light emission portion using the reflection profileinformation.

According to the disclosure, the light emission conditions of the lightemission portion are determined based on the reflection profileinformation, so that a description property to the light emissionconditions that are determined can be ensured.

In the above-described disclosure, the light emission portion may bedisposed between the imaging portion and the object, and have at leastany one of a shape not blocking a visual field at the time of imagingand an optical characteristic not blocking the visual field.

According to the disclosure, by disposing the light emission portionbetween the imaging portion and the object, compared with an occasionthat the light emission portion is not disposed between the imagingportion and the object, an overall compact image processing system canbe provided. As a result, selection of applicable equipment can beavoided from being restricted as much as possible.

According to another example of the disclosure, an image processingdevice which controls an imaging portion imaging an object and a lightemission portion having a light emission surface directed toward theobject to perform an image processing is provided. The image processingdevice includes: a light emission control portion, which controls thelight emission portion in a manner that each of plural types of partialregions set in advance in the light emission surface emits light; animaging control portion, which controls the imaging portion to image insynchronization with light emission of each of the plural types ofpartial regions; and an output portion, which outputs reflection profileinformation, wherein the reflection profile information is obtainedbased on a plurality of images which are captured by the imaging portionin synchronization with the light emission of each of the plural typesof partial regions, and the reflection profile information showsrelationships between positions within the light emission surface anddegrees of light reflected at attention sites of the object and incidentto the imaging portion with respect to the light irradiated to theobject from the positions.

According to the disclosure, the reflection profile information which isa reference in determining the illumination conditions is output, sothat the setting of the illumination conditions becomes easy.

According to another example of the disclosure, an image processingprogram which is executed in an image processing device is provided, andthe image processing device controls an imaging portion imaging anobject and a light emission portion having a light emission surfacedirected toward the object to perform an image processing. The imageprocessing program includes: controlling the light emission portion in amanner that each of plural types of partial regions set in advance inthe light emission surface emits light; controlling the imaging portionto image in synchronization with light emission of each of the pluraltypes of partial regions; and outputting reflection profile information,wherein the reflection profile information is obtained based on aplurality of images which are captured by the imaging portion insynchronization with the light emission of each of the plural types ofpartial regions, and the reflection profile information showsrelationships between positions within the light emission surface anddegrees of light reflected at attention sites of the object and incidentto the imaging portion with respect to the light irradiated to theobject from the positions.

According to the disclosure, the reflection profile information which isa reference in determining the illumination conditions is output, sothat the setting of the illumination conditions becomes easy.

The reflection profile information which is a reference in determiningthe illumination conditions is output, so that the setting of theillumination conditions becomes easy.

§ 1 Application Example

At first, one example of a scene in which the disclosure is applied isFirst, with reference to FIG. 1 and FIG. 2, an example of a scene towhich the present invention is applied will be described. FIG. 1 is adiagram schematically showing an application scene of an imageprocessing system 1 of an embodiment. FIG. 2 is a diagram schematicallyshowing reflection profile information 70.

The image processing system 1 of the embodiment includes a camera 8which is one example of an imaging portion, an illumination device 4which is one example of a light emission portion, and a control device100 which controls processing executed in the image processing system 1.In addition, the control device 100 includes a light emission controlportion 12 which controls the illumination device 4, an imaging controlportion 14 which controls the camera 8, and an output portion 18 whichoutputs the reflection profile information. As an example, the lightemission control portion 12, the imaging control portion 14 and theoutput portion 18 are arranged in the control device 100 which has astructure in accordance with general-purpose computer architecture.

The illumination device 4 has a light emission surface 40 directedtoward an object W. The illumination device 4 can make any region of thelight emission surface 40 emit light, and is made of an organic EL(Electro Luminescence) for example.

The light emission control portion 12 controls the illumination device 4so that each of plural types of partial regions 43 which are set on thelight emission surface 40 emits light. Each of the plural types ofpartial regions 43 at least has different positions within the lightemission surface 40. Sizes of each of the plural types of partialregions 43 may be the same as each other or be different from eachother. In addition, shapes of each of the plural types of partialregions 43 may be the same as each other or be different from eachother. In addition, one portion of one partial region 43 may be a regionin common with one portion of an adjacent partial region 43.

The imaging control portion 14 controls the camera 8 in a manner thatthe object W is imaged in synchronization with light emission of each ofthe plural types of partial regions 43. Here, “the object W is imaged insynchronization” means that the object W is imaged every time the kindof the partial regions 43 emitting light is changed.

The light emission control portion 12 and the imaging control portion 14control the illumination device 4 and the camera 8 as described above,and thereby an input image D is generated for every partial regionemitting light and a plurality of input images D are obtained.

The output portion 18 outputs the reflection profile information 70obtained based on the plurality of input images D. The reflectionprofile information 70 is the information showing relationships betweena position within the light emission surface 40 and degrees of lightreflected at attention sites A of the object W and incident to thecamera 8 with respect to light irradiated to the object W from theaforementioned positions.

Specifically, the reflection profile information 70 is described withreference to FIG. 2. In the example shown in FIG. 2, one kind of partialregion 43 is set in a position (X₁, Y₁) on the light emission surface40. An input image D (X₁, Y₁), which is obtained when the partial region43 set in the position (X₁, Y₁) emits a light, is generated by a lightL_(c) which is resulted from a light L_(i) that is irradiated from theposition (X₁, Y₁) being reflected by the object W and incident to thecamera 8. Accordingly, a feature amount P₁ is obtained from the inputimage D (X₁, Y₁), and the feature amount P₁ is a degree of the lightL_(c) that is reflected at the attention site A of the object W andincident to the camera 8 with respect to the light L_(i) that isirradiated from the position (X₁, Y₁) on the light emission surface 40.Similarly, a feature amount P₂ is obtained from an input image D (X₂,Y₂) which is obtained when a partial region 43 set in a position (X₂,Y₂) emits a light, and the feature amount P₂ is a degree of a lightL_(c) that is reflected at the attention site A of the object W andincident to the camera 8 with respect to the light L_(i) that isirradiated from the position (X₂, Y₂).

Relationships between positions (X, Y) on the light emission surface 40and feature amounts P are obtained from each of the plurality of inputimages D. That is, the reflection profile information 70 is anaggregation of information consisting of the positions (X, Y) on thelight emission surface and the feature amounts P. In the example shownin FIG. 2, the feature amount P₁ to the feature amount P_(N) areobtained for each of the position (X₁, Y₁) to the position (X_(m),Y_(n)) on the light emission surface 40, and the aggregation of thefeature amount P₁ to the feature amount P_(N) is called as thereflection profile information 70.

When the reflection profile information 70 is obtained, a reference ofhow much light is incident to the camera 8 can be known when the lightis irradiated from any position on the light emission surface 40 to theobject W. Therefore, the reflection profile information 70 is areference for determining irradiation conditions. As a result, settingof illumination conditions becomes easy. Furthermore, because thesetting of the illumination conditions can be performed based on thereflection profile information 70, a description property to the setillumination conditions can be ensured.

§ 2 Specific Example A. One Example of Production Line in which ImageProcessing System is Applied

Next, one example of the image processing system of the embodiment isdescribed. FIG. 3 is a schematic diagram showing one example of aproduction line in which the image processing system 1 of the embodimentis applied.

As shown in FIG. 3, the image processing system 1 of the embodimentincludes the camera 8 which images the object W continuously carried in,the illumination device 4 which illuminates the object W, and thecontrol device 100 which controls the illumination device 4 and thecamera 8.

The camera 8 includes an optical system such as a lens, an aperture orthe like and a photoelectric converter such as a CCD (Charge CoupledDevice) image sensor, a CMOS (Complementary Metal Oxide Semiconductor)image sensor or the like as main configuration elements. Thephotoelectric converter is a device which converts light included in animaging visual field 81 of the camera 8 into image signals.

The illumination device 4 irradiates light to the object W disposed on astage 300. Irradiation patterns of the light which are irradiated fromthe light emission surface 40 can be arbitrarily changed according tothe irradiation patterns instructed from the control device 100.

The illumination device 4 has translucency and is typically atranslucent organic EL lamp. When a side where the camera 8 is disposedis set as an upper side and a side where the object W is disposed is setas a lower side with a position where the illumination device 4 isdisposed as a basis, the illumination device 4 may have translucency toa degree that the camera 8 can image, via the illumination device 4, anobject which is positioned lower than the illumination device 4.

The object W, which is an inspection subject, is moved by the movablestage 300 to an inspection position where the camera 8 and theillumination device 4 are fixed. The stage 300 is controlled by a PLC200 (Programmable Logic Controller). The PLC 200 controls the stage 300in a manner that when the object W is conveyed to the inspectionposition, the stage 300 stops instantly until an appearance inspectionby the image processing system 1 is completed. At this moment, thecontrol device 100 irradiates the light to the object W by theillumination device 4 and images the object W by the camera 8. Thecontrol device 100 controls the illumination device 4 to change theirradiation pattern of the light irradiated from the illumination device4, and controls the camera 8 to image the object W by the camera 8 everytime the irradiation pattern of the light is changed. The control device100 inspects an appearance of the object W by using plural pieces ofcaptured images that are obtained as above. In addition, the controldevice 100 outputs an inspection result to the PLC 200 when theappearance inspection is completed. The PLC 200 makes the next object Wbe conveyed to the inspection position based on the output of theinspection result from the control device 100.

The control device 100 is electrically connected to a display portion102 and an input portion 104. The display portion 102 typically consistsof a liquid crystal display and displays setting contents to the userfor example. The input portion 104 typically consists of a mouse andfunctions for inputting information relating to various settings. Forexample, the user can input setting information relating to settings ofinspection conditions by operating the input portion 104 based on theinformation displayed in the display portion 102. Besides, the inputportion 104 is configured by the mouse, and can also be configured by atouch panel, a keyboard or a combination thereof.

B. One Example of Hardware Configuration of Control Device

FIG. 4 is a schematic diagram showing a hardware configuration of thecontrol device 100. The control device 100 includes a CPU (CentralProcessing Unit) 110, a main memory 120, a hard disk 130, a camerainterface (I/F) 180, an illumination I/F 140, a display controller 172,an input I/F 174, a communication I/F 150, an external memory I/F 160and an optical drive 164. Each of these portions is connected to oneanother via a bus 190 so that they are capable of data communication.

The CPU 110 develops a program (a code), which includes an imageprocessing program 132 and a setting program 134 installed on the harddisk 130, in the main memory 120 and executes these programs in apredefined order, thereby performing various calculations. The mainmemory 120 is typically a volatile storage device such as a DRAM(Dynamic Random Access Memory) or the like.

The hard disk 130 is an internal memory included in the control device100 and is a non-volatile storage device. The hard disk 130 includesinspection information 136 relating to the inspection conditions inaddition to the image processing program 132 and the setting program134. Besides, in addition to the hard disk 130 or in place of the harddisk 130, a semiconductor storage device such as a flash memory or thelike can also be adopted.

The camera I/F 180 mediates data transmission between the CPU 110 andthe camera 8. That is, the camera I/F 180 is connected to the camera 8which generates the images. In addition, the camera I/F 180 givescommands of controlling imaging actions in the connected camera 8according to internal commands from the CPU 110.

The illumination I/F 140 mediates the data transmission between the CPU110 and the illumination device 4. That is, the illumination I/F 140 isconnected to the illumination device 4. In addition, the illuminationI/F 140 transmits, according to internal commands from the CPU 110,commands about the irradiation patterns to the connected illuminationdevice 4. The illumination device 4 irradiates the light with theirradiation patterns based on the received commands. Besides, theillumination device 4 may be connected to the control device 100 via thecamera 8. In addition, the camera 8 may be connected to the controldevice 100 via the illumination device 4.

The display controller 172 is connected to the display portion 102 andnotifies the user of a processing result or the like in the CPU 110.That is, the display controller 172 is connected to the display portion102 and controls display in the display portion 102.

The input I/F 174 is connected to the input portion 104 and mediates thedata transmission between the CPU 110 and the input portion 104. Thatis, the input I/F 174 receives operation commands given by the useroperating the input portion 104. The operation commands include, forexample, operation commands for setting the inspection conditions.

The communication I/F 150 exchanges various data between the PLC 200 andthe CPU 110. Besides, the communication I/F 150 can also exchange databetween a server and the CPU 110. The communication I/F 150 includeshardware corresponding to a network for exchanging various data with thePLC 200.

The external memory I/F 160 is connected to an external memory 6 andperforms processing of reading/writing of data into the external memory6. The external memory 6 is detachable to the control device 100 and istypically a non-volatile storage device such as a USB (Universal SerialBus) memory, a memory card or the like. In addition, various programssuch as the image processing program 132 or the like are not required tobe saved in the hard disk 130, and can also be saved in a server whichis communicable with the control device 100, the external memory 6 whichcan be directly connected to the control device 100, or an optical disk164A. For example, the various programs which are executed in thecontrol device 100 and various parameters used in the various programsare circulated in a state of being stored in the external memory 6, andthe external memory I/F 160 reads out the various program and thevarious parameters from the external memory 6. Elsewise, the programs orthe parameters which are downloaded from the server or the like which iscommunicably connected to the control device 100 may be installed on thecontrol device 100. In addition, the control device 100 can also installthe programs or the parameters on the control device 100 from theoptical disk 164A through the optical drive 164.

The optical drive 164 reads out from the optical disk 164A or the likethe various programs stored therein and installs the various programs onthe hard disk 130.

Besides, the image processing program 132 and the setting program 134 ofthe embodiment can also be provided by being incorporated into oneportion of other programs. In addition, alternatively, part of or all ofthe functions provided by the execution of the image processing program132 can also be implemented as a dedicated hardware circuit.

C. Structure of Illumination Device 4

FIG. 5 is a schematic diagram in which one portion of the illuminationdevice 4 of the embodiment is enlarged. The illumination device 4includes a plurality of illumination elements 41 which are disposed in amatrix shape. The illumination device 4 can light up each illuminationelement 41 independently. The irradiation patterns in the embodiment aredetermined by the illumination elements 41 to be lighted up among theplurality of illumination elements 41. In addition, in the embodiment,it is described that white light is irradiated from each illuminationelement 41, and the irradiation patterns refer to shading patterns onthe light emission surface 40. In addition, in the embodiment, theillumination device 4 can light up or light off each illuminationelement 41 independently. Besides, the illumination device 4 can alsoadjust light emission intensity of each illumination element 41.

Each illumination element 41 includes, for example, a light emissionregion and a translucent region, and by making the light emission regionemit light, it can be approximated as that the entire illuminationelement 41 emits light in terms of an illumination effect to the objectW. In addition, the translucency can be kept by having the translucentregion.

D. First Specific Example

In the following, reflection profile information 72 in the firstspecific example is described.

<Obtainment Method of Reflection Profile Information 72>

The reflection profile information 72 is the information showingrelationships between positions within the light emission surface 40 anddegrees of the light reflected at the attention sites A of the object Wand incident to the camera 8 with respect to the light irradiated to theobject W from the aforementioned positions. FIG. 6 is a diagram showingone example of a process for obtaining the reflection profileinformation 72 in the first specific example. The process for obtainingthe reflection profile information 72 includes a process Ph1 forobtaining the data, a process Ph2 for extracting the data, and a processPh3 for generating the reflection profile information 72 from theextracted data.

(Process Ph1 for Obtaining Data)

FIG. 7 is a diagram for describing an obtainment method of data.Besides, in FIG. 7, an illustration of the camera 8 is omitted. Theillumination device 4 can make only the partial regions 43 set withinthe light emission surface 40 emit light. The light emission surface 40consists of an aggregation of all the illumination elements 41. On theother hand, the partial regions 43 consist of one or a plurality ofillumination elements 41. That is, the partial regions 43 are a partialaggregation with respect to the aggregation of all the illuminationelements 41. The illumination device 4 can light up each of theplurality of illumination elements 41 independently, and thus theillumination device 4 can make only the set partial regions 43 emitlight by lighting up all the illumination elements 41 included withinthe partial regions 43 set on the light emission surface 40. Besides,the illumination device 4 may make all the illumination elements 41included in the partial regions 43 lighted up with the same lightemission intensity or with light emission intensities different from oneanother.

The control device 100 controls the illumination device 4 to light upeach of plural types of partial regions 43 set on the light emissionsurface 40. Each of the plural types of partial regions 43 is differentat least in positions within the light emission surface 40. Here, thepositions of the partial regions 43 within the light emission surface 40mean centres of the partial regions 43. A size of each of the pluraltypes of partial regions 43 may be the same as each other or bedifferent from each other. In addition, a shape of each of the pluraltypes of partial regions 43 may be the same as each other or bedifferent from each other. In addition, one portion of one partialregion 43 may be a region in common with one portion of an adjacentpartial region 43. In the embodiment, the shape and the size of each ofthe plural types of partial regions 43 are the same as one another, andeach partial region 43 does not overlap with another partial region 43.Sizes of the partial regions 43 may be the sizes that can ensure a lightamount to a degree at which the camera 8 can capture images from whichat least one portion of the object W can be recognized when the partialregions 43 are made to emit light. Here, because the irradiationpatterns refer to the shading patterns of the light emission surface 40,making one partial region 43 emit light can be referred to as making thelight emission surface 40 emit light with one irradiation pattern. Inaddition, each of the plural types of partial regions 43 can be referredto as an irradiation pattern different from one another.

In an example shown in FIG. 7, the control device 100 sets the partialregions 43 in each of the position (X₁, Y₁) to the position (X_(m),Y_(n)) on the light emission surface 40 and makes a total of m×n partialregions 43 emit light independently. The control device 100 controls thecamera 8 to image in synchronization with the light emission of each ofthe plural types of partial regions 43, and obtains a total of m×n inputimages D. Here, in the embodiment, for the sake of convenience, an Xdirection on the light emission surface 40 is described as a movementdirection of the object W in FIG. 3, and a Y direction is described as adirection orthogonal to the X direction and an irradiation direction.The input images D (X, Y) mean the images captured and obtained in astate when the partial regions 43 set in the positions (X, Y) are madeto emit light. Here, in FIG. 7, a position of each of the camera 8, theillumination device 4 and the object W is fixed. That is, the imagingconditions at the time when the input image D (X₁, Y₁) to the inputimage D (X_(m), Y_(n)) are captured are different at least in thepositions on the light emission surface 40 which emits light.

(Process Ph2 for Extracting Data)

FIG. 8 is a diagram for describing an extraction method of the data. Thecontrol device 100 extracts partial images M corresponding to theattention sites A of the object W from each of the input image D (X₁,Y₁) to the input image D (X_(m), Y_(n)). Here, the partial images Mcorresponding to the attention sites A of the object W are the partialregions which are extracted to include attention positions a (x, y)corresponding to the attention sites A of the object W within the inputimages D. All the input image D (X₁, Y₁) to the input image D (X_(m),Y_(n)) are obtained by being captured in the same imaging visual field81. Therefore, camera coordinate positions (x, y) of the partial imagesM corresponding to the attention sites A are in common in each of theinput image D (X₁, Y₁) to the input image D (X_(m), Y_(n)).

The partial images M may be data configured by one pixel or be dataconfigured by a plurality of pixels, and may include at least pixels ofthe attention positions a (x, y) corresponding to the attention sites A.Besides, the attention sites A may show certain prescribed positions orshow predefined ranges.

The control device 100 extracts the partial images M of the attentionpositions a (x, y) within a camera coordinate corresponding to theattention sites A from each of the input images D (X₁, Y₁) to D (X_(m),Y_(n)), and obtains the partial image M (X₁, Y₁|x, y) to the partialimages M (X_(m), Y_(n)|x, y). Here, the left of a vertical bar inbrackets of the partial images M (X, Y|x_(r), y_(r)) means the positions(X, Y) of the light emission surface 40 which emits light when the inputimages D which are extraction origins are captured. The right of thevertical bar in the brackets of the partial images M (X, Y|x_(r), y_(r))means the attention positions a (x, y) within the input images D whichare targets at the time of extracting the partial images M. That is, thepartial images M (X, Y|x_(r), y_(r)) mean the partial regions of theinput images D (X, Y) which are extracted to include attention positionsa_(r) (x_(r), y_(r)) within the input images D (X, Y).

(Process Ph3 for Generating Reflection Profile Information 72 fromExtracted Data)

FIG. 9 is a diagram for describing a generation method of the reflectionprofile information 72. The control device 100 extracts feature amountsp from each of the partial image M (X₁, Y₁|x, y) to the partial image M(X_(m), Y_(n)|x, y). The feature amounts p are values showingintensities of the light which are reflected at the attention sites Aand incident to the camera 8 among the light which are incident to theattention sites A from the partial regions 43 set on the light emissionsurface 40, and include, for example, luminance values, or valuesobtained by standardizing the luminance values. In addition, the featureamounts p can also be information obtained after performing apre-processing such as a spatial filtering or the like on the partialimages M. Here, the feature amounts p (x, y|X, Y) mean feature amountsextracted from the partial images M (X, Y|x, y).

In the embodiment, an aggregation of the feature amount p (x, y|X₁, Y₁)to the feature amount p (x, y|X_(m), Y_(n)) is the reflection profileinformation 72. That is, the control device 100 generates the reflectionprofile information 72 by extracting the feature amount p (x, y|X₁, Y₁)to the feature amount p (x, y|X_(m), Y_(n)) from the partial image M(X₁, Y₁|x, y) to the partial image M (X_(m), Y_(n)|x, y).

Here, the left of the vertical bar in the brackets of the featureamounts p (x, y|X, Y) means the attention positions a (x, y) which arepositions of the attention sites A within the imaging visual field 81.The right of the vertical bar in the brackets of the feature amounts p(x, y|X, Y) means which positions (X, Y) on the light emission surface40 are made to emit light to obtain the feature amounts p. That is, thefeature amounts p (x, y|X, Y) can also be referred to as the valuesbased on the light incident from the positions (X, Y) on the lightemission surface 40, which are among the light incident to the attentionsites A (the attention positions a (x, y)).

In addition, the input images D (X, Y) are generated by the lightincident to the camera 8 among the light which are irradiated from thepartial regions 43 set in the positions (X, Y) on the light emissionsurface 40 and reflected at the attention sites A. In addition, thefeature amounts p (x, y|X, Y) are values showing intensities of thelight that reflected at the attention sites A and incident to the camera8 among the light incident to the object W from the positions (X, Y) onthe light emission surface 40. That is, the feature amounts p (x, y|X,Y) can be referred to as the information showing degrees of the lightreflected at the attention sites A and incident to the camera 8 withrespect to the light irradiated from the positions (X, Y). Therefore,the reflection profile information 72, which is configured by theaggregation of the feature amount p (x, y|X₁, Y₁) to the feature amountp (x, y|X_(m), Y_(n)), is referred to as the information which shows therelationships between the positions (X, Y) within the light emissionsurface 40 which are the irradiation positions, and the degrees of thelight reflected at the attention sites A and incident to the camera 8with respect to the light irradiated from the irradiation positions.

<Output Method of Reflection Profile Information 72>

FIG. 10 is a diagram showing one example of an output method of thereflection profile information 72. In the example shown in FIG. 10, thecontrol device 100 outputs a mapping image 542, in which the featureamounts p are mapped in an XY coordinate which is the coordinate systemon the light emission surface 40, as the reflection profile information72. The mapping image 542 is an image in which gray values 56 shown bythe shading degrees corresponding to a magnitude of the feature amountsp are mapped in the XY coordinate. For example, the gray value 56 (x,y|X₁, Y₁) corresponding to the magnitude of the feature amount p (x,y|X₁, Y₁) is mapped in a position (X₁, Y₁) of the XY coordinate. Thecontrol device 100 converts each of the feature amount p (x, y|X₁, Y₁)to the feature amount p (x, y|X_(m), Y_(n)) into the gray values 56 andobtains the gray value 56 (x, y|X₁, Y₁) to the gray value 56 (x,y|X_(m), Y_(n)). The control device 100 generates the mapping image 542by mapping each of the gray values 56 (x, y|X₁, Y₁) to 56 (x, y|X_(m),Y_(n)) in the XY coordinate. Besides, in FIG. 10, in order to make iteasy to understand how the gray values 56 are mapped, the gray values 56are shown as images having a predefined area (the same applies to FIG.11).

The control device 100 outputs the reflection profile information 72 bydisplaying the mapping image 542 on the display portion 102. Besides, anoutput destination of the mapping image 542 is, but not limited to, forexample, the display portion 102 connected to the control device 100,and can also be a printer, the PLC 200, or a portable terminal or thelike.

The feature amounts p are values showing intensities of the lightreflected at the attention sites A and incident to the camera 8 amongthe light incident to the object W from the partial regions 43 set onthe light emission surface 40. Therefore, the user can intuitivelygrasp, based on the mapping image 542, effects of the light irradiatedfrom which irradiation position are strong when the images of theattention sites A are generated. For example, in a case that the imageshave darker colours when the feature amounts p are greater, the user canintuitively grasp from the mapping image 542 that, images includinggreat feature amounts can be obtained by making the partial regions 43which are set in the positions mapped by the images with darker coloursemit light.

<Determination of Illumination Conditions Using Reflection ProfileInformation 72>

The reflection profile information 72 is the information showing therelationships between the positions within the light emission surface 40and the degrees of the light reflected at the attention sites A of theobject W and incident to the camera 8 with respect to the lightirradiated to the object W from the positions. Therefore, when theillumination conditions at the time of performing an image measurementare set, it can be predicted from the reflection profile information 72which direction it is from which the light is incident to the attentionsites A to obtain images suitable as the images used in the imagemeasurement. In the following, description is made on a setting methodof the illumination conditions using the reflection profile information72 when the image measurement of the attention sites A is performed.

FIG. 11 is a diagram showing one example of a user interface screen 600during the setting. The user interface screen 600 shown in FIG. 11 isdisplayed in the display portion 102 by selecting an illuminationsetting tab 610. Besides, in FIG. 10, although tabs other than theillumination setting tab 610 are omitted, the tabs relating to othersettings such as a “camera setting” tab for setting the camera 8, an“image processing setting” tab for setting the processing contentsexecuted on the obtained images and so on may also be arranged.

The user interface screen 600 includes an image display region 620, anedition region 630, a tool selection region 640, a display selectionregion 650 and an instruction region 660.

The image display region 620 is a region which displays the image of theimaging visual field 81 captured by the camera 8. The user operates amouse or the like, which is one example of the input portion 104, andselects the attention position a corresponding to the attention site Afrom the image displayed on the image display region 620. In FIG. 11, aregion including the selected attention site A is displayed on the imagedisplay region 620 as a selection region 622. Besides, the attentionsite A may show a certain prescribed position or show a predefinedrange. That is, the attention position a corresponding to the attentionsite A may be configured by one pixel or be configured by a plurality ofpixels. Besides, the selection region 622 is desirable to be a shapecorresponding to a range of the attention position a. For example, whenthe attention position a is configured by one pixel, the selectionregion 622 is shown by a point, and when the attention position a isconfigured by a plurality of pixels, the selection region 622 is shownby a region of a predefined range.

The edition region 630 is a region in which the mapping image 542 of theattention position a which is selected by an operation to the imagedisplay region 620 can be displayed, and is a region for setting theillumination conditions based on the displayed mapping image 542. Thatis, the display which shows the position (the attention position a) ofthe attention site A within the imaging visual field 81 and the mappingimage 542 of the attention site A which is selected by the operation tothe image display region 620 are displayed on the same user interfacescreen 600. Therefore, the user can easily grasp the mapping image 542being displayed is the mapping image 542 with respect to the attentionsite A in which position (attention position a) within the imagingvisual field 81.

The coordinate system of the mapping image 542 is a coordinate systemshowing the light emission surface 40. That is, the coordinate system ofthe edition region 630 is also the coordinate system showing the lightemission surface 40. For example, the user can operate the input portion104 to use a square tool 642, a sectorial tool 644, a triangular tool646 or the like displayed in the tool selection region 640 to plot theregion (the light emission region) which emits light within the editionregion 630 displayed in the coordinate system showing the light emissionsurface 40. At this moment, a light emission region 632 is plotted inthe mapping image 542. In other words, when a designation of the lightemission region is received from the input portion 104, the receivedlight emission region is reflected in the mapping image 542. Besides,although the light emission region which is a light emission range isdesignated to the edition region 630, as long as the light emissionregion can be designated to the coordinate system on the light emissionsurface 40, the designated light emission range may not be displayed onthe mapping image 542. The designation of the light emission range meansdesignating the light emission intensities for every position on thelight emission surface 40. Besides, in FIG. 11, an example is shown inwhich the designation of whether the light is emitted is received as thelight emission intensities, and the light emission intensities can alsobe adjusted.

By reflecting the designated light emission range on the mapping images542, when the images of the attention positions a are obtained by theirradiation patterns in which the designated light emission range islighted up, the light emission range can be designated while confirmingthe images showing what kind of feature amounts p are obtained.

In addition, as shown in FIG. 11, the image display region 620 and theedition region 630 are regions adjacent to one another. That is, theimage of the imaging visual field 81 and the image of the mapping image542 are displayed side by side. Therefore, the user can easily grasp themapping image 542 is the mapping image 542 with respect to the attentionsite A of which position (attention position a) within the imagingvisual field 81.

The tool selection region 640 is a region for selecting the tool (thesquare tool 642, the sectorial tool 644 and the triangular tool 646)used for plotting the light emission region 632. The user can select thetool by moving the cursor to the tool to be used for plotting the lightemission region 632 and clicking. The user can use the selected tool todesignate the light emission range.

The display selection region 650 is a region for determining contentsdisplay in the edition region 630. In the display selection region 650,a mapping display tab 652 and an irradiation pattern display tab 654 arearranged. If the mapping display tab 652 is selected, the mapping image542 of the attention site A which is selected in the image displayregion 620 is displayed in the edition region 630. If the irradiationpattern display tab 654 is selected, the irradiation pattern which isset for the attention site A selected in the image display region 620 isdisplayed in the edition region 630. The selection may be performed toboth of the mapping display tab 652 and the irradiation pattern displaytab 654, and if both are selected, the mapping image 542 and an image(the light emission region 632) which shows the region of a registeredlight emission range are overlapped and displayed in the edition region630.

The instruction region 660 includes an obtainment button 662, an imagingbutton 664, a register button 666 and a back button 668.

The obtainment button 662 is a button for making each of the pluraltypes of partial regions 43 emit light and generating the reflectionprofile information 72 by the above-described method. If it is detectedthat the obtainment button 662 is selected, the control device 100executes the processing for generating the reflection profileinformation 72. In addition, when the obtainment button 662 is selected,the attention site A (the attention position a) is selected in the imagedisplay region 620, and when the mapping display tab 652 is selected inthe display selection region 650, the control device 100 updates thedisplay of the edition region 630 to the mapping image 542 for theattention site A which is selected.

The imaging button 664 is a button for taking images in a state that therange within the light emission surface 40 set in the edition region 630is made to emit light. The control device 100 instructs, based on thedetection that the imaging button 664 is selected, the illuminationdevice 4 to irradiate the light by the irradiation patterns which makethe light emission region 632 displayed in the edition region 630 emitlight, and instructs the camera 8 to take images under the irradiationpatterns.

The register button 666 is a tab for registering the irradiationpatterns which make the light emission region 632 displayed in theedition region 630 emit light as the irradiation patterns of theattention site A (the attention position a shown by the selection region622) selected in the image display region 620. The back button 668 is abutton for cancelling without registering the irradiation patternsdisplayed in the edition region 630. When the back button 668 isselected, for example, the user interface screen 600 is deleted from thedisplay portion 102.

In this way, the user can set the attention position a within theimaging visual field 81, and set the illumination conditionscorresponding to reflection characteristics of the attention site Acorresponding to the selected attention position a. For example, whenany region on the light emission surface 40 is made to emit light by themapping image 542 displayed in the display portion 102, the user canrecognize whether images including or not including the feature amountsp are obtained, and can set the illumination conditions for obtainingthe images including or not including the feature amounts p via theinput portion 104. The control device 100 associates the irradiationpatterns which are the set illumination conditions with the attentionpositions a of the attention sites A within the imaging visual field 81for storage. Besides, the control device 100 may also associates theirradiation patterns which are the set illumination conditions with theposition of the attention sites A within the object W for storage, andprescribe the position (the attention positions a) within the imagingvisual field 81 from an arrangement situation of the object W within theimaging visual field 81 and the position of the attention sites A withinthe object W.

(One Example of Image Measurement Method)

The user can set a plurality of attention positions a within the imagingvisual field 81 and set irradiation patterns L which are theillumination conditions according to the reflection characteristics ofthe attention sites A corresponding to each of the plurality ofattention positions a. With reference to FIG. 12, a generation method ofan inspection image 51 which is generated using the irradiation patternsL is described. Here, the inspection image 51 is an image used in theimage measurement executed by the image processing system 1. FIG. 12 isa diagram for describing one example of the generation method of theinspection image 51.

The control device 100 controls the illumination device 4 tosequentially change the irradiation patterns of the light irradiate fromthe illumination device 4, and controls the camera 8 to image the objectW under each irradiation pattern. The control device 100 performs theappearance inspections of the object W based on the inspection image 51generated from the plurality of images which are captured in each of theplurality of irradiation patterns.

The irradiation patterns L are set for each attention position a (x, y)within the imaging visual field 81 of the camera 8. Here, (x, y) meanscoordinate values of the attention positions a within the imaging visualfield 81. The inspection image 51 used in the appearance inspections isgenerated from a plurality of original images 52 which are captured andobtained respectively under each irradiation pattern L. The images ofthe attention positions a within the inspection image 51 are generatedfrom the original images 52 which are captured under the irradiationpatterns L set in association with the attention positions a.

In the example shown in FIG. 12, the attention position a₁ (x₁, y₁) tothe attention position a_(n) (x_(p), y_(q)) are set as the attentionpositions a within the imaging visual field 81. The irradiation patternsL are set for every attention position a. The control device 100 obtainsthe original image 52 (x₁, y₁) to the original image 52 (x_(p), y_(q))which are captured under the irradiation pattern L₁ to the irradiationpattern L_(n).

The control device 100 generates the inspection image 51 from theplurality of original image 52 (x₁, y₁) to the original image 52 (x_(p),y_(q)) that are obtained. The control device 100 generates an image ofthe attention position a1 (x₁, y₁) within the inspection image 51 basedon an extraction image 53 (x₁, y₁) which includes the position (x₁, y₁)within the original image 52 (x₁, y₁). Similarly, the control device 100generates an image of the attention position a₂ (x₂, y₁) within theinspection image 51 based on the extraction image 53 (x₂, y₁), andgenerates an image of the attention position a_(n) (x_(p), y_(q)) withinthe inspection image 51 based on the extraction image 53 (x_(p), y_(q)).

In other words, the inspection image 51 is generated from the extractionimage 53 (x₁, y₁) to the extraction image 53 (x_(p), y_(q)). Pixelsincluded in the extraction images 53 may be one pixel or a plurality ofpixels. Ranges of the extraction images 53 are set corresponding todistances between the attention positions a and the attention positionsadjacent to the attention positions a, and are set in a manner that oneinspection image 51 is generated from the extraction image 53 (x₁, y₁)to the extraction image 53 (x_(p), y_(q)). Besides, the ranges of theextraction images 53 can also be set so that the extraction images 53overlap with one another. On this occasion, pixel information ofoverlapped parts is generated based on more than two overlappedextraction images 53.

The irradiation patterns L set for the attention positions a are setaccording to the reflection characteristics of the attention sites A ofthe object corresponding to the attention positions a. Therefore, theattention sites A can be measured based on the images captured under theillumination conditions suitable for the reflection characteristics ofthe attention sites A. In this way, images can be captured under theillumination conditions which are suitable for the reflectioncharacteristics of each of a plurality of attention sites A on theobject W, so that the image measurement with a high precision can beperformed.

(Function Configuration of Control Device 100)

FIG. 13 is a diagram schematically showing a function configuration ofthe control device 100. The control device 100 controls eachconfiguration shown in FIG. 13 to set the irradiation patterns byexecuting the setting program 134.

The control device 100 includes, for example, the light emission controlportion 12, the imaging control portion 14, a detection portion 11, anextraction portion 16 and the output portion 18.

The detection portion 11 detects the designation of the position (x_(r),y_(r)) within the imaging visual field 80, which is input from the inputportion 104. For example, if the obtainment button 662 shown in FIG. 10is operated, the detection portion 11 judges whether the input images D,which are necessary for obtaining the mapping image 542, are saved inthe main memory 120, and if not saved, the light emission controlportion 12 and the imaging control portion 14 are notified of the factthat an obtainment instruction has been made to execute the process Ph1for obtaining the input images D. The light emission control portion 12and the imaging control portion 14 function as obtainment portions forexecuting the process Ph1.

The light emission control portion 12 controls, according to theobtainment instruction, the illumination device 4 to make the pluralityof partial regions 43 which is set on the light emission surface 40sequentially emit light.

The imaging control portion 14 controls the camera 8 to sequentiallygenerate the images of the imaging visual field 81 corresponding to thesequential light emission of the partial regions 43. The imaging controlportion 14 associates the input images D generated by the camera 8 withthe positions (X, Y) on the light emission surface 40 of the partialregions 43 which emit light when the input images D are captured, andsaves the association in the main memory 120 which is on kind of astorage portion.

When it is judged that the input images D which are necessary forobtaining the mapping image 542 are saved in the main memory 120, thedetection portion 11 instructs the extraction portion 16 to extract thepartial images M which are corresponding to the position (x_(r), y_(r))input from the input portion 104 (an extraction instruction). Theposition (x_(r), y_(r)) input from the input portion 104 is a positionof the attention position a within the imaging visual field 81, and isthe position (x, y) of the selection region 622 within the image displayregion 620.

The extraction portion 16 extracts, according to the extractioninstruction, the partial images M which are corresponding to thepositions of the camera coordinate (x, y) from each of the plurality ofinput images D saved in the main memory 120. If the partial images M areextracted, the extraction portion 16 instructs the output portion 18 tooutput the mapping image 542 for the designated attention position a (anoutput instruction).

The output portion 18 outputs, according to the output instruction, thereflection profile information 72 as the mapping images 542, and in thereflection profile information 72, the feature amounts p shown by thepartial images M is associated with the positions (X, Y) of the partialregions 43 which emit light when the input images D being extractionorigins of the partial images M are generated. Specifically, the outputportion 18 displays the mapping image 542 on the display portion 102,wherein the mapping image 542 is mapped with the gray values 56 (x, y|X,Y) at the corresponding positions (X, Y) on the XY coordinate, and thegray values 56 (x, y|X, Y) corresponds to the magnitude of the featureamounts p (x, y|X, Y) which are shown by the partial images M (X, Y|x,y) of the attention positions a (x, y) extracted from the input images D(X, Y) corresponding to the positions (X, Y).

The control device 100 may further include a determination portion 13that determines the irradiation patterns for every attention position aand stores the irradiation patterns in the hard disk 130 as inspectioninformation 136. If it is detected that the register button 666 of FIG.10 is operated, the detection portion 11 instructs the determinationportion 13 to register the irradiation patterns (register instruction).

The determination portion 13 associates, at a timing when the registerbutton 666 is operated, the attention positions a (x, y) designated inthe image display region 620 with the irradiation patterns L set in theedition region 630 for a determination and stores the association as theinspection information 136 in the hard disk 130.

The control device 100 includes an imaging measurement portion 15 forperforming the image measurement. The detection portion 11 detects thata signal (an inspection start signal) which shows that the object W isconveyed to the predefined inspection position has been notified fromthe PLC 200. If the inspection start signal is detected, the detectionportion 11 instructs the imaging measurement portion 15 to start theimage measurement (a measurement instruction).

The imaging measurement portion 15 performs the image measurement inaccordance with the inspection information 136 stored in the hard disk130. Specifically, the imaging measurement portion 15 instructs theimaging control portion 14 in a manner that the irradiation patterns Lis set in the light emission control portion 12 in accordance with theinspection information 136 and to generate the extraction images 53 ofthe positions (x, y) corresponding to the attention positions a whichare corresponding to the set irradiation patterns L. Accordingly, theimaging measurement portion 15 obtains a plurality of extraction images53, generates the inspection image 51 based on the obtained extractionimages 53 and measures an appearance of the object W based on thegenerated inspection image 51. Besides, the inspection image 51 isgenerated from a plurality of extraction images 53, so that it can alsobe said that the imaging measurement portion 15 performs the imagemeasurement of a region including the attention positions a based on theextraction images 53 of the attention positions a corresponding to theirradiation patterns L. An image measurement result that is obtained istransmitted to the PLC 200 or the like.

<Flowchart>

FIG. 14 is a flowchart of the processing of outputting the reflectionprofile information 72 of the designated attention position a. Theprocessing of outputting the reflection profile information 72 is aprocessing which is implemented by the CPU 110 developing the settingprogram 134 in the main memory and executing the setting program 134.The processing of outputting the reflection profile information 72 isexecuted every time the obtainment button 662 is operated.

In step S11, the CPU 110 judges whether the input images D are saved inthe main memory 120. When it is judged that the input images D are notsaved in the main memory 120 (“NO” in step S11), the CPU 110 switchesthe processing to step S12.

In step S12, the CPU 110 judges whether all the partial regions 43 areset on the light emission surface 40. All the partial regions 43 meanall of the plural types of partial regions 43 set in advance. Forexample, the CPU 110 judges whether each partial region 43 which is setin different positions on the light emission surface 40 is set to allthe positions that are set. When all of the plural types of partialregions 43 emit light, the plural types of partial regions 43 may be setin a manner that the range on the light emission surface 40 of thepredefined range emits light. The predefined range is a rangearbitrarily set according to a size of the object W, the imaging visualfield 81 or the like. When it is judged that there are partial regions43 which are not set on the light emission surface 40 (“NO” in stepS12), the CPU 110 switches the processing to step S13.

In step S13, the partial regions 43 are set on the light emissionsurface 40 and the set partial regions 43 are caused to emit light.

In step S14, the CPU 110 makes the camera 8 generate the input images D.The CPU 110 repeats step S13 and step S14 until it is judged to be “YES”in step S12. Accordingly, the CPU 110 can make each of the plural typesof partial regions 43 emit light, and can generate the input images D insynchronization with the light emission of each of the plural types ofpartial regions 43.

When it is judged that the input images D are saved in the main memory120 (“YES” in step S11), or it is judged that all the partial regions 43are set on the light emission surface 40 (“YES” in step S12), the CPU110 switches the processing to step S15.

In step S15, the CPU 110 extracts the partial images M which arecorresponding to the designated attention positions a from each of aplurality of input images D.

In step S16, the CPU 110 outputs the reflection profile information 72in which the feature amounts p shown by the partial images M areassociated with the positions of the partial regions 43 which emit lightwhen the input images D being the extraction origins of the partialimages M are generated.

E. Second Specific Example

The reflection profile information 72 in the first specific example isthe information in association with the positions on the light emissionsurface 40. The reflection profile information 74 in the second specificexample is different from the reflection profile information 70 in thatthe reflection profile information 74 is information in association withrelative positions (i, j) with respect to the positions (the attentionpositions a) within the imaging visual field of the attention sites A.Besides, the imaging method of the input images D, the extraction methodof the partial images M and the extraction method of the feature amountsp are in common with the reflection profile information 70, so that thedescription is partially omitted.

(About Relative Positions)

The relative positions are described. The relative positions arerepresented by an ij coordinate. FIG. 15 is a diagram for describing therelative positions. Specifically, the ij coordinate is a coordinatewhich takes the position on the light emission surface 40 having acorrespondence relationship with the attention position a as a referenceposition. Here, the reference position means, for example, an origin (0,0). For example, a relationship of a formula (1) is established betweenthe attention position a_(r) (x_(r), y_(r)) of the attention site A_(r)within the camera coordinate system and a corresponding position (X_(r),Y_(r)) on the light emission surface 40. Therefore, the correspondingposition (X_(r), Y_(r)) on the light emission surface 40 can be obtainedfrom the attention position a_(r).

$\begin{matrix}\left( {{Expression}\mspace{14mu} 1} \right) & \; \\{\begin{pmatrix}X_{r} \\Y_{r}\end{pmatrix} = {{A\begin{pmatrix}x_{r} \\y_{r}\end{pmatrix}} + B}} & (1)\end{matrix}$

Coefficients A, B are calibration parameters, and can be calculated bycalculation based on a position relationship between the camera 8 andthe illumination device 4 after the positions of the camera 8 and theillumination device 4 are fixed, or obtained by performing a calibrationoperation. Besides, the formula (1) is one example, and the positions onthe light emission surface 40 which are corresponding to the positionswithin the camera coordinate system can also be defined in advancewithout using the formula (1).

<Calibration Method>

One example of a calibration method for obtaining the correspondencerelationships between the attention positions a and the positions (X, Y)on the light emission surface 40 is described. The attention positions acorrespond to the positions (x, y) within the imaging visual field 81,so that the correspondence relationships between the positions (X, Y) onthe light emission surface 40 and the positions (x, y) within theimaging visual field 81 are obtained by calibration.

The control device 100 sequentially sets the partial regions 43 todifferent positions on the light emission surface 40 and makes thepartial regions 43 emit light, and controls the camera 8 to sequentiallygenerate the images of the imaging visual field 81 corresponding to thesequential light emission. In the inspection position, a referenceobject called a target plate for the calibration may be disposed toperform the calibration, or the object W which is an inspection targetmay be disposed to perform the calibration.

The control device 100 extracts the luminance values from each of theplurality of pixels included in the plurality of images which isobtained corresponding to the sequential lighting up. The control device100 compares the luminance values of the pixels positioned in (x, y)within the plurality of images, and prescribes the pixels which have thehighest luminance values. The control device 100 associates the position(X, Y) where the partial regions 43 are set when the imagescorresponding to the prescribed pixels are obtained with the positions(x, y) of the pixels within the images. The control device 100 canobtain correspondence relationships between the camera coordinate systemand the illumination coordinate system by performing the same processingto all the pixels within the obtained images. Calibration parameters maybe calculated by making the correspondence relationships between thecamera coordinate system and the illumination coordinate system linearlyapproximated. Besides, the processing described above is performed forevery pixel, and the processing described above can also be performedfor every plural pixels taking the plural pixels as one unit.

Because the calibration is performed as described above, the lightirradiated from the positions (X, Y) on the light emission surface 40corresponding to the attention positions a can be said to be the lighthaving the highest light amount reflected at the attention positions aand incident to the camera 8 among the light incident to the attentionpositions a.

<Generation of Reflection Profile Information 74>

FIG. 16 is diagram showing a generation method of the reflection profileinformation 74 in the second specific example. Besides, the obtainmentmethods of the input images D, the partial images M, and the featureamounts p which are used for generating the reflection profileinformation 74 are in common with the first specific example, so thatthe description is partially omitted.

The range of the feature amounts p which are included in the reflectionprofile information 74 in the second specific example is set to a rangeshown in a formula (2).

(Expression 2)

{i|−{tilde over (ω)}≤i≤{tilde over (ω)}}

{j|−{tilde over (ω)}≤j≤{tilde over (ω)}}  (2)

In an example shown in FIG. 16, the reflection profile information 74 ofthe attention position a_(r) (x_(r), y_(r)) within the imaging visualfield 81 is generated. The control device 100 generates the input imagesD included in the range shown in the formula (3), and extracts a partialimage M of the attention position a_(r) (x_(r), y_(r)) from each inputimage D. Specifically, the control device 100 extracts the partial imageM of the attention position a_(r) (x_(r), y_(r)) from each of the inputimage D (X_(r)−w, Y_(r)−w) to the input image D (X_(r)+w, Y_(r)+w), andextracts the partial image M (X_(r)−w, Y_(r)−w|x_(r), y_(r)) to thepartial image M (X_(r)+w, Y_(r)+w|x_(r), y_(r)). Besides, the inputimage D (X_(r)−w, Y_(r)−w) to the input image D (X_(r)+w, Y_(r)+w) canalso be obtained by extracting the input images D included in the rangeshown in the formula (3) from the input image D (X₁, Y₁) to the inputimage D (X_(m), Y_(n)).

(Expression 3)

{X|X _(r) −{tilde over (ω)}≤X≤X _(r)+{tilde over (ω)}}

{Y|Y _(r) −{tilde over (ω)}≤Y≤Y _(r)+{tilde over (ω)}}  (3)

Here, the range shown in the formula (3) is defined based on the formula(2). In addition, the input images D of the range shown in the formula(3) mean the input images D which are obtained when the partial regions43 set within the range shown in the formula (3) emit light.

The control device 100 extracts the feature amounts p from each of thepartial image M (X_(r)−w, Y_(r)−w|x_(r), y_(r)) to the partial image M(X_(r)+w, Y_(r)+w|x_(r), y_(r)) to generate the reflection profileinformation 74. Here, if the XY coordinate is converted to the ijcoordinate, the feature amount p (x_(r), y_(r)|X_(r)−w, Y_(r)−w) to thefeature amount p (x_(r), y_(r)|X_(r)+w, Y_(r)+w) can be represented asfeature amount p (a_(r)|−w, −w) to the feature amount p (a_(r)|w, w).The feature amount p (a_(r)|−w, −w) can also be referred to as a valuebased on the light which are among the light incident to the attentionsite A_(r) and incident from the position (−w, −w) within the ijcoordinate taking the position (X_(r), Y_(r)) on the light emissionsurface 40 corresponding to the attention position a_(r) (x_(r), y_(r))as the reference position.

The reflection profile information 74 is different from the reflectionprofile information 72 in the first specific example in that thereflection profile information 74 is represented not by the XYcoordinate on the light emission surface 40, but by the ij coordinatetaking the positions on the light emission surface 40 which havecorrespondence relationships with the attention positions a as thereference positions. Because the reflection profile information 74 isrepresented by the ij coordinate, the reflection profile information 74which reflects changes of the position relationship between the lightemission surface 40 and the attention sites A can be obtained. As aresult, the reflection profile information 74 obtained for everyattention site A can be compared with one another in the same dimensionregardless of the position relationships between the light emissionsurface 40 and the attention sites A.

In addition, the range of the feature amounts p included in thereflection profile information 74 is set to the range shown by theformula (2), so that the reflection profile information 74 obtained forevery attention site A is the information in the range the same with oneanother. As a result, the control device 100 can directly compare thereflection profile information 74 obtained for every attention site Awith one another.

<Output Method of Reflection Profile Information 74>

FIG. 17 is a diagram showing an example of outputting the reflectionprofile information 74 in the second specific example as a mapping image544. The control device 100 maps the gray values 56, which arecorresponding to the magnitude of the feature amounts p shown by each ofthe plural partial images M that are extracted, in the ij coordinatetaking the position (X_(r), Y_(r)) as the origin. For example, the grayvalue 56 (a_(r)|−w, −w) of the feature amount p (a_(r)|−w, −w) is mappedto (−w, −w) within the ij coordinate. Besides, in FIG. 17, in order tomake it easy to understand how the gray values 56 are mapped, the grayvalues 56 are shown as images with predefined areas (the same applies toFIG. 20 and FIG. 21(a) to FIG. 21(d)).

<Determination Method of Illumination Conditions Using ReflectionProfile Information 74>

In the first specific example, the example is shown in which thereflection profile information 70 is used in order to set differentirradiation patterns L for each of the plurality of attention positionsa set within the imaging visual field 81. The reflection profileinformation 74 in the second specific example is used for setting areference irradiation pattern L₀. The reference irradiation pattern L₀is an irradiation pattern which is a reference used to determine theirradiation pattern L of every attention position a. In the following, adetermination method of the irradiation pattern L of every attentionposition a using the reference irradiation pattern L₀ is described, anda setting method of the reference irradiation pattern L₀ is described.

(Method for Determining Irradiation Patterns L from ReferenceIrradiation Pattern L₀)

The control device 100 sets the irradiation patterns L by setting alight emission region shown by the reference irradiation pattern L₀ inthe positions on the light emission surface 40, which have thecorrespondence relationships in advance with the attention positions a,when the extraction images 53 corresponding to the attention positions aare obtained.

FIG. 18 is a diagram for describing the determination method of theirradiation pattern L of every attention position a using the referenceirradiation pattern L₀. The irradiation pattern L_(r) is determined sothat the irradiation pattern L₀ is made taking the position (X_(r),Y_(r)) on the light emission surface 40 corresponding to the attentionposition a_(r) (x_(r), y_(r)) as a centre. Specifically, when a functionshowing a shape (the light emission range) of the reference irradiationpattern L₀, which is the reference, is defined as L₀(i, j), arelationship of a formula (4) is established between the irradiationpattern L_(r) and the reference irradiation pattern L₀.

(Expression 4)

L ₀(i,j)=L _(r)(X−X _(r) ,Y−Y _(r))  (4)

The control device 100 can calculate the irradiation pattern L_(r) fromthe attention position a_(r) (x_(r), y_(r)), the reference irradiationpattern L₀, the formula (1) and the formula (4).

FIG. 19 is a diagram for describing a generation method of theinspection image 51 using the reference irradiation pattern L₀. Thepositions (X, Y) on the light emission surface 40 corresponding to theattention positions a are set as reference positions to set the lightemission range shown by the reference irradiation pattern L₀, and theoriginal images 52 are generated, and the images of the positions whichare corresponding to the attention positions a on the inspection image51 are generated based on the extraction images 53 extracted from theoriginal images 52. That is, the light emission range shown by thereference irradiation pattern L₀ is also sequentially set in thedifferent positions on the light emission surface 40 by sequentiallychanging the attention positions a.

(Setting Method of Reference Irradiation Pattern L₀)

FIG. 20 is a diagram showing a user interface screen 700 at the time ofsetting the reference irradiation pattern L₀. Compared with the userinterface screen 600 shown in FIG. 11, the user interface screen 700 isdifferent in that the user interface screen 700 includes an editionregion 730 in place of the edition region 630, and includes an imagingbutton 764 in place of the imaging button 664.

The mapping images 544 of the ij coordinate system are displayed in theedition region 730. If the register button 666 is operated, a lightemission region 732 which is set in the edition region 730 is saved asthe reference irradiation pattern L₀.

If the imaging button 764 is operated, the inspection image 51, which iscaptured and generated under the reference irradiation pattern L₀ shownby the light emission region 732 plotted within the edition region 730,is displayed in the image display region 620. Accordingly, the user canjudge whether the required image is obtained by the referenceirradiation pattern L₀ that is set.

FIG. 21(a) to FIG. 21(d) are diagrams showing one example of a flow ofsetting the reference irradiation pattern L₀. The object W shown in FIG.21(a) to FIG. 21(d) is a metal with machining marks formed on a surface.In the example shown in FIG. 21(a) to FIG. 21(d), the input images D arealready obtained. In addition, in FIG. 21(a) to FIG. 21(d), only theimage display region 620 and the edition region 730 of the userinterface screen 700 are representatively shown.

Scratches to be detected in the image measurement in the attention siteA₁ of the object W are set to be the scratches on which the machiningmarks which are not desired to be detected in the image measurement inthe attention site A₂ are formed. The user operates the input portion104 to select, based on the images of the object W displayed within animage display region 620, the attention position a₁ corresponding to theattention site A₁. If the attention position a₁ is selected, as shown inFIG. 21(a), the control device 100 displays the mapping image 544corresponding to the attention position a₁ in the edition region 730.The user plots the region (the light emission region 732) which emitslight within the edition region 730 displayed by the ij coordinatesystem.

In the example shown in FIG. 21(a) to FIG. 21(d), a darker colour isshown when the magnitude shown by the feature amounts p is greater, andit is judged that there are scratches if the magnitude shown by thefeature amounts p is great. Because the scratches to be detected areformed in the attention position a₁, great feature amounts p aredesirably extracted from the attention position a₁ in the imagemeasurement. Therefore, if the user plots the light emission region 732to include a part shown by a dark colour, the illumination conditionunder which the marks like the attention site A₁ is easily detected canbe set. In FIG. 21(b), an example in which the user plots the lightemission region 732 to include the part shown by the dark colour isshown.

Here, when the machining marks like the attention site A₂ are imaged,the irradiation patterns under which small feature amounts p areextracted are desirable. If the user operates the input portion 104 toselect, based on the images of the object W displayed within the imagedisplay region 520, the attention position a₂ corresponding to theattention site A₂ in a state shown in FIG. 21(b), a state shown in FIG.21(c) is switched to.

In the edition region 730 shown in FIG. 21(c), the light emission region732 which is set to easily detect the scratches is plotted in themapping image 544 corresponding to the attention position a₂. The useradjusts, as shown in FIG. 21(d), the light emission region 732 for theedition region 730 shown in FIG. 21(c) so as not to include the partshown by the dark colour.

In this way, the user can set the illumination conditions under whichthose to be detected as scratches can be detect and those not to bedetected are not detected in the image measurement.

<Function Configuration Diagram of Control Device 100 in Second SpecificExample>

FIG. 22 is a diagram schematically showing a function configuration ofthe control device 100 in the second specific example. The controldevice 100 may include an extraction range determination portion 17 inaddition to the function configuration shown in FIG. 13.

The extraction range determination portion 17 determines a range ofextraction to create the mapping images 544 in the reference coordinatesystem (i, j) which takes the position (X_(r), Y_(r)) on the lightemission surface 40 corresponding to a camera coordinate (x_(r), y_(r))that is input as a reference position. Specifically, the extractionrange determination portion 17 prescribes the position (X_(r), Y_(r)) onthe light emission surface 40 corresponding to the camera coordinate(x_(r), y_(r)) based on corresponding information 138 showingcorrespondence relationships between camera coordinate positions and thepositions on the light emission surface 40, which are stored in astorage portion such as the hard disk 130 or the like. The correspondinginformation 138 includes, for example, the calibration parameters. Theextraction range determination portion 17 determines a range(X_(r)−w<X<X_(r)+w, Y_(r)−w<Y<Y_(r)+w) to be extracted with theprescribed position (X_(r), Y_(r)) as the reference position.

The extraction portion 16 extracts the partial images M in the cameracoordinate (x_(r), y_(r)) from each of the input images D included inthe extraction range (X_(r)−w<X<X_(r)+w, Y_(r)−w<Y<Y_(r)+w). Whenextracting the partial images M, the extraction portion 16 instructs theoutput portion 18 to output the reflection profile information 74 withrespect to the designated attention position a_(r) (x_(r), y_(r)) as themapping image 544 (output instruction).

The output portion 18 outputs, according to the output instruction, thefeature amounts p shown by the partial images M by associating thepositions (X, Y) of the partial regions 43, which emit light when theinput images D being the extraction origins of the partial images M aregenerated, with the relative positions (i, j). The output portion 18outputs the reflection profile information 74 by displaying, in thedisplay portion 102, the mapping images 544 in which the gray values 56corresponding to the magnitude of the feature amounts p shown by thepartial images M are mapped in the ij coordinate which takes thepositions on the light emission surface 40 having the correspondencerelationships with the attention positions a as the reference positions.

In addition, if the register button 666 of FIG. 20 is operated, thereference irradiation pattern L₀ set in the edition region 630 is storedas the inspection information 136 during the operation of the registerbutton 666.

The control device 100 may further include a calibration portion 19. Thecalibration portion 19 generates the corresponding information 138. Ifthe detection portion 11 detects an instruction of calibration, anexecution of calibration is instructed to the calibration portion 19.

The calibration portion 19 instructs the light emission control portion12 and the imaging control portion 14 to obtain the input images D, setsthe positions on the light emission surface 40 corresponding to eachpixel position (x, y) from the obtained input images D and stores thepositions as the corresponding information 138 in the hard disk 130 orthe like.

In addition, compared with the function configuration shown in FIG. 13,the control device 100 is different in that the control device 100includes an imaging measurement portion 15 a in place of the imagingmeasurement portion 15. The imaging measurement portion 15 a performsthe image measurement in accordance with the inspection information 136stored in the hard disk 130. Specifically, the imaging measurementportion 15 a makes the light emission control portion 12 set a lightemission range shown by the reference irradiation pattern L₀ indifferent positions (X, Y) on the light emission surface 40 inaccordance with the inspection information 136, and instructs theimaging control portion 14 to generate extraction images 53 of thepositions (x, y) within the camera coordinate corresponding to thepositions (X, Y) on the light emission surface 40 to which the lightemission range is set. In this way, the imaging measurement portion 15obtains a plurality of extraction images 53, generates the inspectionimage 51 based on the obtained extraction image 53 and measures theappearance of the object W based on the inspection image 51 that isgenerated. The image measurement result that is obtained is transmittedto the PLC 200 or the like.

<Flowchart in Second Specific Example>

FIG. 23 is a flowchart of the processing of outputting the reflectionprofile information 74 in the second specific example of the designatedattention position a. The flowchart shown in FIG. 23 is different fromthe flowchart shown in FIG. 14 in that step S15-1 is arranged.

In step S15-1, the CPU 110 sets the extraction range. The CPU 110designates a predefined range as the extraction range with the positionson the light emission surface 40 corresponding to the designatedattention positions a as the reference position. In the following, instep S15, the partial images M are extracted in accordance with theextraction range designated in step S15-1.

F. Variation Example of Position Relationship Among Illumination Device,Camera and Object

In the embodiment, the example in which the illumination device 4 isdisposed between the camera 8 and the object W is shown. Besides, theposition relationship among the illumination device 4, the camera 8 andthe object W is not limited to be coaxial, and may be a positionrelationship in which the light from the illumination device 4 areirradiated to the object W and at least one portion of the object W isincluded in the imaging visual field 81 of the camera 8.

FIG. 24 is a diagram showing a variation example of the positionrelationship among the illumination device 4, the camera 8 and theobject W. For example, the illumination device 4 may not be disposed onan optical axis of the camera 8. In the example shown in FIG. 24, theillumination device 4 is not disposed on the optical axis of the camera8, so that an illumination device without permeability can be adopted asthe illumination device 4.

G. Variation Example of Illumination Device

In the embodiment, a translucent organic EL lamp is mentioned as oneexample of the illumination device 4. Besides, when the illuminationdevice 4 is disposed between the camera 8 and the object W, theillumination device 4 may have at least any one of a shape not blockingthe visual field at least during the imaging and an opticalcharacteristic not blocking the visual field. For example, theillumination device 4 may be the illumination device 4 in which anopening is arranged in one portion, the illumination device 4 in whichone portion is configured by a coaxial incident illumination, or thelike. FIG. 25 is a diagram showing a variation example of theillumination device. For the illumination device 400 of the variationexample, one portion is configured by a coaxial incident illumination410 and the other portion is configured by an illumination 420 withouttranslucency.

H. Variation Example of Method for Outputting Reflection ProfileInformation 70

In the embodiment, as an output method of the reflection profileinformation 70, a method of outputting the reflection profileinformation 70 in which the feature amounts p are mapped to thepredefined coordinate system is exemplified. However, for example, thereflection profile information 70 can also be output by an approximateformula or the like in which the relationships between the positions onthe light emission surface 40 and the feature amounts p are expressed.

In addition, the mapping images are set as two-dimensional plane images,but the reflection profile information 70 can also be represented bythree or more dimensions. For example, a representation form can also beused in which the feature amounts p are plotted in a coordinate systemwhich consists of a coordinate axis showing the magnitude of the featureamounts p in addition to the coordinate axes corresponding to thepositions on the light emission surface 40.

I. Output Method of Mapping Image 544 of Every Attention Position

In the second specific example, the mapping images 544 are sequentiallyswitched as one example of the mapping images 544 obtained for each ofthe plurality of attention positions, but the plurality of mappingimages 544 can also be overlapped and displayed.

J. Variation Example of Setting Method of Partial Regions 43

In the embodiment, each partial region 43 is set not to be overlappedwith another partial region 43. Besides, each partial region 43 can alsobe set to be overlapped with another partial region 43 in one portion.For example, by setting each partial region 43 to be overlapped withanother partial region 43, a light amount at the time of making onepartial region 43 emit light can be ensured without reducing aresolution of the reflection profile information.

In addition, when the light emission intensity of each illuminationelement 41 can be adjusted and each partial region 43 is set to beoverlapped with another partial region 43, the illumination device 4 canalso make the partial regions 43 emit light in a manner that the lightemission intensities of the illumination elements 41 increase fromboundaries of the partial regions 43 towards centres. On this occasion,the shading pattern of the partial regions is in the form of Gaussiandistribution. In this way, compared with an occasion when the partialregions 43 evenly emit light, the light amount at the time of making onepartial region 43 emit light can be further increased without reducingthe resolution of the reflection profile information.

K. Generation Method of Reflection Profile Information when AttentionSites a Show Prescribed Ranges

In the embodiment, the attention sites A show the prescribed positions,and the attention positions a corresponding to the attention sites A areinformation showing one point. Besides, the attention sites A may alsobe information showing prescribed ranges, and on this occasion, theattention positions a may also be ranges a′ showing certain rangeswithin the images corresponding to the prescribed ranges. On thisoccasion, the reflection profile information of the attention sites Amay be generated, for example, based on the reflection profileinformation obtained for each of the plurality of attention positions aincluded in the range a. For example, the reflection profile informationof the attention sites A may be the information in which the reflectionprofile information obtained for each of the plurality of attentionpositions a is standardized, or be the information which is expressed bya representation form in which a plurality of mapping images obtainedwhen each reflection profile information is mapped to the ij coordinateis synthesized. In addition, the reflection profile information of theattention sites A can also be obtained by setting the ranges of thepartial images M to the ranges corresponding to the ranges a.

L. Operation and Effect

As described above, both the reflection profile information 72 in thefirst specific example and the reflection profile information 74 in thesecond specific example (also referred to as the reflection profileinformation 70 in general hereinafter), which are output by the controldevice 100, are the information which shows the relationships betweenthe positions (X, Y) within the light emission surface 40 and thefeature amounts p showing the degrees of the light reflected at theattention sites A of the object W and incident to the camera 8 withrespect to the light irradiated to the object W from the positions (X,Y).

Therefore, if the light is irradiated from any position on the lightemission surface 40 to the object W based on the reflection profileinformation 70, a rough standard of how much light is incident to thecamera 8 can be known, and the reflection profile information 70 becomesreference information for determining the illumination conditions. As aresult, the setting of the illumination conditions becomes easy.

In addition, the feature amounts p are obtained from luminanceinformation within the input images D which are corresponding to theattention positions a within the imaging visual field 81. In addition,the reflection profile information 74 in the second specific example isthe information associated with the relative positions (i, j) withrespect to the positions (the attention positions a) within the imagingvisual field of the attention sites A. Therefore, regardless of theposition relationships between the light emission surface 40 and theattention sites A, the reflection profile information 74 obtained forevery attention site A can be compared with one another in the samedimension.

The control device 100 outputs the reflection profile information 74 tothe ij coordinate system corresponding to the relative positions in arepresentation form of the mapping images 544 to which the featureamounts p are output. Therefore, because the reflection profileinformation 74 is represented by the same coordinate system regardlessof the position relationships between the light emission surface 40 andthe attention sites A, the reflection profile information 74 can beeasily compared when compared for every attention site A.

The attention positions a can be designated for the object W displayedin the image display region 620, and the mapping images 544 of thedesignated attention positions a are displayed in the edition region 630or the edition region 730. Therefore, the user can easily grasp themapping image 544 is related to which attention site A of the positionwithin the imaging visual field 81.

As shown in FIG. 21(a) to FIG. 21(d), the user can set a plurality ofattention positions a for the images of the object W which are displayedin the image display region 620, and can sequentially display themapping images of the plurality of attention positions a that is set inthe edition region. Accordingly, the mapping images for every attentionposition a can be compared easily. Besides, in the example shown in FIG.21(a) to FIG. 21(d), an example is shown in which one attention positiona is selected and one mapping image 544 is displayed; however, aplurality of attention positions a may be simultaneously selected andthe mapping images 544 corresponding to the plurality of attentionpositions a that is selected may be simultaneously or sequentiallyoutput.

The irradiation patterns L are determined based on the reflectionprofile information 72. Therefore, the description property to thedetermined illumination condition can be ensured. Besides, similarly,the description property about the reference irradiation pattern L₀determined based on the reflection profile information 74 can also beensured.

Compared with an occasion that the illumination device 4 cannot bedisposed between the camera 8 and the object W, an overall compact imageprocessing system 1 can be provided by using the illumination device 4which can be disposed between the camera 8 and the object W as theillumination device. As a result, restriction on selection of applicableequipment can be avoided as far as possible.

M. Appendix

As described above, the embodiment includes disclosures as below.

(Configuration 1)

An image processing system including:an imaging portion (8), which images an object (W);a light emission portion (4, 400), which includes a light emissionsurface (40) directed toward the object (W);a light emission control portion (12), which controls the light emissionportion (4, 400) in a manner that each of plural types of partialregions (43) set in advance within the light emission surface (40) emitslight;an imaging control portion (14), which controls the imaging portion (8)to image in synchronization with light emission of each of the pluraltypes of partial regions (43); andan output portion (18), which outputs reflection profile information(70, 72, 74), wherein the reflection profile information (70, 72, 74) isobtained based on a plurality of images (D) which are captured by theimaging portion (8) in synchronization with the light emission of eachof the plural types of partial regions (43), and the reflection profileinformation (70, 72, 74) shows relationships between positions (X, Y, i,j) within the light emission surface (40) and degrees of light (L_(c))reflected at attention sites of the object (W) and incident to theimaging portion (8) with respect to light (L_(i)) irradiated to theobject (W) from the positions.

(Configuration 2)

The image processing system according to configuration 1, wherein thereflection profile information (74) is information which is obtainedfrom each of the plurality of images (D) and which is based on luminanceinformation (p) corresponding to attention points (a) in an imagingvisual field within the images (D) and relative positions (i, j) of thepartial regions (43), in which light is emitted when the images arecaptured with respect to the attention points (a).

(Configuration 3)

The image processing system according to configuration 2, wherein theoutput portion (18) outputs the reflection profile information (74) by arepresentation form (544) in which the information (56) corresponding tothe luminance information is output to a coordinate system with two ormore axes corresponding to the relative positions (i, j).

(Configuration 4)

The image processing system according to configuration 3, wherein theoutput portion (18) outputs the reflection profile information (74) bythe representation form (544, 730), wherein the reflection profileinformation (74) corresponds to the attention points (a) that aredetermined based on the position information on images (620) which aredesignated by a user with respect to the images (620) of the imagingvisual field (81) captured by the imaging portion (8).

(Configuration 5)

The image processing system according to configuration 3 or 4, whereinthe output portion (18) simultaneously or sequentially outputs thereflection profile information (74) obtained for a plurality ofattention points (a) within the imaging visual field (81) to thecoordinate system (i, j) (730).

(Configuration 6)

The image processing system according to any one of configurations 1 to5, further including a determination portion (13) which determines lightemission conditions (L) of the light emission portion (4, 400) using thereflection profile information (72, 74).

(Configuration 7)

The image processing system according to any one of configurations 1 to6, wherein the light emission portion (4, 400) is disposed between theimaging portion and the object, and has at least any one of a shape notblocking a visual field at the time of imaging and an opticalcharacteristic not blocking the visual field.

(Configuration 8)

An image processing device (100), which controls an imaging portion (8)imaging an object (W) and a light emission portion (4, 400) having alight emission surface (40) directed toward the object (W) to perform animage processing, including:a light emission control portion (12), which controls the light emissionportion (4, 400) in a manner that each of plural types of partialregions (43) set in advance in the light emission surface (40) emitslight;an imaging control portion (14), which controls the imaging portion (8)to image in synchronization with light emission of each of the pluraltypes of partial regions (43); and an output portion (18), which outputsreflection profile information (70, 72, 74), wherein the reflectionprofile information (70, 72, 74) is obtained based on a plurality ofimages (D) which are captured by the imaging portion (8) insynchronization with the light emission of each of the plural types ofpartial regions (43), and the reflection profile information (70, 72,74) shows relationships between positions (X, Y, i, j) within the lightemission surface (40) and degrees of light (L_(c)) reflected atattention sites (A) of the object (W) and incident to the imagingportion (8) with respect to light (L_(i)) irradiated to the object (W)from the positions.

(Configuration 9)

An image processing program (134), which is executed in an imageprocessing device (100) that controls an imaging portion (8) imaging anobject (W) and a light emission portion (4, 400) having a light emissionsurface (40) directed toward the object (W) to perform an imageprocessing, the image processing program (134) including:a step (S13), in which the light emission portion (4, 400) is controlledin a manner that each of plural types of partial regions (43) set inadvance in the light emission surface (40) emit light;a step (S14), in which the imaging portion (8) is controlled to image insynchronization with light emission of each of the plural types ofpartial regions (43); anda step (S16), in which reflection profile information (70, 72, 74) isoutput, wherein the reflection profile information (70, 72, 74) isobtained based on a plurality of images (D) which are captured by theimaging portion (8) in synchronization with the light emission of eachof the plural types of partial regions (43), and the reflection profileinformation (70, 72, 74) shows relationships between positions (X, Y, i,j) within the light emission surface (40) and degrees of light (L_(c))reflected at attention sites of the object (W) and incident to theimaging portion (8) with respect to light (L_(i)) irradiated to theobject (W) from the positions.

(Configuration 10)

An image processing system, including:an imaging portion (8), which images an object (W);a light emission portion (4), which is disposed between the imagingportion (8) and the object (W), and has a light emission surface (40)directed toward the object (W);a light emission control portion (12), which controls the light emissionportion (4) to make unit partial regions (43) with predefined sizessequentially be set in different positions (X, Y) on the light emissionsurface (40) to emit light;an imaging control portion (14), which controls the imaging portion tosequentially generate, corresponding to the sequential light emission ofthe unit partial region (43), input images (D) which are images of animaging visual field;an extraction portion (16), which extracts, corresponding to designationof attention positions (a) with respect to the imaging visual field,partial images (M) which are corresponding to the attention positions(a) from at least one portion of the input images (D) sequentiallygenerated; andan output portion (18), which associates feature amounts (P) that areshown by the partial images (M) extracted by the extraction portion (16)with positions (X, Y) of the unit partial regions (43) which emit lightwhen the input images (D) being extraction origins of the partial images(M) are generated and outputs the feature amounts (P).

(Configuration 11)

The image processing system according to configuration 10, wherein thefeature amounts (P) are values showing intensities of light reflected atthe attention positions (a) and incident to the imaging portion amonglight incident to the attention positions (a) from the light unitpartial regions (43) set on the emission surface (40).

(Configuration 12)

The image processing system according to configuration 10 or 11, furtherincluding a display portion (102),wherein the output portion (18) makes first images (542, 544) displayedin the display portion (102), and in the first images (542, 544), thepositions (X, Y) of corresponding unit partial regions (43) on a firstcoordinate system (XY, ij) showing the light emission surface (40) arerepresented by a display form in accordance with the magnitude of thefeature amounts (P) corresponding to the positions of the unit partialregions (43).

(Configuration 13)

The image processing system according to configuration 12, furtherincluding:an input portion (104), which receives, in the first coordinate system(XY, ij) displayed in the display portion (102), designation of lightemission intensities for every position on the light emission surface(40); anda determination portion (13), which determines the irradiation patterns(L) used in image measurement of the object (W) corresponding to thedesignation of the input portion (104).

(Configuration 14)

The image processing system according to configuration 13, wherein theoutput portion (18) reflects the designation of the input portion (104)to the first images (542, 544).

(Configuration 15)

The image processing system according to configuration 13 or 14, whereinthe images (620) of the imaging visual field which are captured by theimaging portion (8) are displayed in the display portion (102);the input portion (104) receives user designations (622, 662) withrespect to the images (620) of the imaging visual field which aredisplayed in the display portion (102); andthe extraction portion (16) prescribes the attention positions (a) basedon the user designations (622, 662) received by the input portion.

(Configuration 16)

The image processing system according to configuration 15, wherein theimages (620) of the imaging visual field captured by the imaging portionand the first images (542, 544) are displayed side by side.

(Configuration 17)

The image processing system according to any one of configurations 13 to16,wherein the determination portion (13) determines the irradiationpatterns (L) in association with the designated attention positions (a),andan imaging measurement portion (15) is further included which performsthe image measurement of the region including the attention positions(a) based on images (53) of the attention positions (a) corresponding tothe irradiation patterns (L), which are generated by the imaging portion(8) when the light emission surface (40) emits light by the irradiationpatterns (L) determined by the determination portion (13).

(Configuration 18)

The image processing system according to any one of configurations 13 to16,wherein the light emission portion (4) makes the light emission regionshown by the irradiation pattern (L₀) determined by the determinationportion (13) sequentially set in different positions on the lightemission surface (40) and emit light, and the imaging portion (8)sequentially generates the images (53) corresponding to the positions(x, y) within the imaging visual field, which have correspondencerelationships with the positions on the light emission surface (40)which are set corresponding to the sequential light emission of thelight emission region (8), andan imaging measurement portion (15 a) is further included which performsthe image measurement of the object based on a plurality of imagesgenerated by the imaging portion (8) corresponding to the sequentiallight emission of the light emission region.

(Configuration 19)

The image processing system according to configuration 18, wherein theextraction portion (16) extracts the partial images (M) from the inputimages (D), which are generated when the unit partial region emitslight, and the unit partial region are set in a position included withina predefined range (X_(r)−w≤X≤X_(r)+w, Y_(r)−w≤Y≤Y_(r)+w) which takesthe position (X_(r), Y_(r)) on the light emission surface havingcorrespondence relationship with the designated attention position(a_(r)) as a reference position.

It should be considered that the embodiment disclosed here isillustrative instead of limitative in all aspects. The scope of thedisclosure is shown by the claims instead of the description above andmeanings equivalent to the claims and all modifications within the scopeare intended to be included in the scope of the disclosure. In addition,the disclosures described in the embodiments and each variation exampleare intended to be performed individually or in combination whereverpossible.

What is claimed is:
 1. An image processing system, comprising: animaging portion, which images an object; a light emission portion, whichcomprises a light emission surface directed toward the object; a lightemission control portion, which controls the light emission portion in amanner that each of plural types of partial regions set in advance inthe light emission surface emits light; an imaging control portion,which controls the imaging portion to image in synchronization withlight emission of each of the plural types of partial regions; and anoutput portion, which outputs reflection profile information, whereinthe reflection profile information is obtained based on a plurality ofimages which are captured by the imaging portion in synchronization withthe light emission of each of the plural types of partial regions, andthe reflection profile information shows relationships between positionswithin the light emission surface and degrees of light reflected atattention sites of the object and incident to the imaging portion withrespect to the light irradiated to the object from the positions.
 2. Theimage processing system according to claim 1, wherein the reflectionprofile information is information which is obtained from each of theplurality of images and which is based on luminance informationcorresponding to attention points of an imaging visual field in theimages and relative positions of the partial regions in which the lightis emitted when the images are captured with respect to the attentionpoints.
 3. The image processing system according to claim 2, wherein theoutput portion outputs the reflection profile information by arepresentation form in which the information corresponding to theluminance information is output to a coordinate system with two or moreaxes corresponding to the relative positions.
 4. The image processingsystem according to claim 3, wherein the output portion determines theattention points based on the position information on the images whichare designated by a user with respect to the images of the imagingvisual field captured by the imaging portion, and outputs the reflectionprofile information corresponding to the determined attention points bythe representation form.
 5. The image processing system according toclaim 3, wherein the output portion simultaneously or sequentiallyoutputs the reflection profile information obtained for each of aplurality of attention points within the imaging visual field to thecoordinate system.
 6. The image processing system according to claim 4,wherein the output portion simultaneously or sequentially outputs thereflection profile information obtained for each of a plurality ofattention points within the imaging visual field to the coordinatesystem.
 7. The image processing system according to claim 1, furthercomprising a determination portion which determines light emissionconditions of the light emission portion using the reflection profileinformation.
 8. The image processing system according to claim 2,further comprising a determination portion which determines lightemission conditions of the light emission portion using the reflectionprofile information.
 9. The image processing system according to claim3, further comprising a determination portion which determines lightemission conditions of the light emission portion using the reflectionprofile information.
 10. The image processing system according to claim4, further comprising a determination portion which determines lightemission conditions of the light emission portion using the reflectionprofile information.
 11. The image processing system according to claim5, further comprising a determination portion which determines lightemission conditions of the light emission portion using the reflectionprofile information.
 12. The image processing system according to claim6, further comprising a determination portion which determines lightemission conditions of the light emission portion using the reflectionprofile information.
 13. The image processing system according to claim1, wherein the light emission portion is disposed between the imagingportion and the object, and has at least one of a shape not blocking avisual field at the time of imaging and an optical characteristic notblocking the visual field.
 14. The image processing system according toclaim 2, wherein the light emission portion is disposed between theimaging portion and the object, and has at least one of a shape notblocking a visual field at the time of imaging and an opticalcharacteristic not blocking the visual field.
 15. The image processingsystem according to claim 3, wherein the light emission portion isdisposed between the imaging portion and the object, and has at leastone of a shape not blocking a visual field at the time of imaging and anoptical characteristic not blocking the visual field.
 16. The imageprocessing system according to claim 4, wherein the light emissionportion is disposed between the imaging portion and the object, and hasat least one of a shape not blocking a visual field at the time ofimaging and an optical characteristic not blocking the visual field. 17.The image processing system according to claim 5, wherein the lightemission portion is disposed between the imaging portion and the object,and has at least one of a shape not blocking a visual field at the timeof imaging and an optical characteristic not blocking the visual field.18. The image processing system according to claim 6, wherein the lightemission portion is disposed between the imaging portion and the object,and has at least one of a shape not blocking a visual field at the timeof imaging and an optical characteristic not blocking the visual field.19. An image processing device, which controls an imaging portionimaging an object and a light emission portion having a light emissionsurface directed toward the object to perform an image processing,comprising: a light emission control portion, which controls the lightemission portion in a manner that each of plural types of partialregions set in advance in the light emission surface emits light; animaging control portion, which controls the imaging portion to image insynchronization with light emission of each of the plural types ofpartial regions; and an output portion, which outputs reflection profileinformation, wherein the reflection profile information is obtainedbased on a plurality of images which are captured by the imaging portionin synchronization with the light emission of each of the plural typesof partial regions, and the reflection profile information showsrelationships between positions within the light emission surface anddegrees of light reflected at attention sites of the object and incidentto the imaging portion with respect to the light irradiated to theobject from the positions.
 20. An image processing program, which isexecuted in an image processing device that controls an imaging portionimaging an object and a light emission portion having a light emissionsurface directed toward the object to perform an image processing, theimage processing program comprising: controlling the light emissionportion in a manner that each of plural types of partial regions set inadvance in the light emission surface emits light; controlling theimaging portion to image in synchronization with light emission of eachof the plural types of partial regions; and outputting reflectionprofile information, wherein the reflection profile information isobtained based on a plurality of images which are captured by theimaging portion in synchronization with the light emission of each ofthe plural types of partial regions, and the reflection profileinformation shows relationships between positions within the lightemission surface and degrees of light reflected at attention sites ofthe object and incident to the imaging portion with respect to the lightirradiated to the object from the positions.